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DEVELOPMENT OF THE HYBRID APPROACH TO DATA PROCESSING 


H. A. Weeden, F. Y. Borden, D. N. Applegate, and N. B. Bolling 


In response to a request by MITRE Corporation 1 , ORSER evaluated 
two approaches to ERTS-1 MSS data analysis: photointerpretation of 

imagery, and computer processing of digital tapes. Subsequently, ORSER 
combined these two methods, developing the hybrid approach to ERTS data 
processing. This method has been successfully applied in many of the 
research projects conducted by ORSER personnel in the past year. The 
development of the hybrid approach to data processing is discussed here. 

A site consisting of 144 square miles surrounding Harrisburg, 
Pennsylvania, was chosen for study of land use categories* Initially, 
two separate research teams studied the two forms of ERTS-1 data, 
imagery and digital computer tapes. Weeden and Bolling concentrated 
their efforts on using photointerpretive techniques on ERTS imagery, 
while Borden and Applegate analyzed the digital data using only USGS 
7 1/2 minute quadrangle maps as a reference. As the objective at this 
preliminary stage was to determine the extent to which these two dif- 
ferent approaches might succeed individually in mapping land use, there 
was no interaction between the two teams during the first few weeks of 
the study. 


Photointerpretation of ERTS-1 Imagery 


The imagery used in the photointerpretation study was that of 
September 6, 1972, covering the Harrisburg area, namely image number 
1045-15243 in the four channels of the multispectral scanner (MSS). 

This ERTS scene was chosen as the best representation of the study area 
available at the time. The photointerpretation was carried out inde- 
pendent of outside aid. There was no ground truth study of the test 


1 0RSER performed this work under Mitre Purchase Order N35490, as 
directed by Edward A. Ward. For the sake of convenience, ORSER used the 
same test area to work out the combined techniques, although this phase 
of the project was not under the sponsorship of the Mitre Corporation. 
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area, no coordination with other researchers using computer programs, 
and no previous study of maps or aerial photos at larger scales. The 
intent was to determine what could be read directly from ERTS imagery 
alone. Although both interpreters had a traveler's acquaintance with 
the Harrisburg area, care was taken not to identify items by their geo- 
graphic location. Graytone variations were recorded, but interpreted 
only where their shape provided interpretive clues. 

The imagery was studied under the following conditions: 

1. Direct inspection of the image on a light table under 
magnifications of 4.5X and 7X, using a direct viewing lens or one lens 
of an Old Delft stereoscope. 

2. Projection of the image by means of a Visucom overhead 
projector, from 10 ft, onto a flat screen at a magnification of 4X. 

3. Projection of the image onto a table by means of a single 
Kelsh Plotter projector, at a magnification of 4.5X. 

" 4. Projection of a glossy positive 4X enlargement using a 
Saltzman projector, resulting in a further enlarged scale of 7.5X (or 
2 miles to the inch). 

The above systems were, as far as we know, the only ones available at 
The Pennsylvania State University at the time. Working at contact scale 
proved useless for documentation, although considerable detail could be 
observed with the hand lens. The overhead projector also could not be 
used, as the projected image could be viewed clearly only from a posi- 
tion of several feet from the screen. The Kelsh Plotter was second only 
to the Saltzman in usefulness. It permitted direct projection of the 
image onto a table, where features could be mapped as observed. However, 
only a very small portion of the image could be viewed at one time, 
making it difficult to determine significant graytone signatures and to 
maintain consistency in delineating them. Mapping by this method was, 
in addition, a very slow process. The Saltzman projector appeared to 
give the best overall image definition combined with rapid tracing of 
observed features. It's chief drawback was the necessity of using 
photographic prints rather than the images themselves, resulting in 
some loss of graytone resolution. 

Positive glossy prints of the portion of the ERTS scene covering 
the test area were made for use in the Saltzman projector. Only a small 
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.portion of the site (approximately 36 square miles) was chosen for study, 
as this portion was considered to be sufficient to illustrate the prob- 
lems involved and the results obtainable by photointerpretive techniques. 
The time involved in producing land use maps from these prints, using 
the Saltzman, varied from 1 to 2 1/2 hours, depending on the channel. 
Channel 5 took the longest, and channels 6 and 7 the shortest, time to 
map. The results were of such quality that it was not considered worth- 
while to attempt to planimeter the areas for quantification of the land 
use categories. Table 1 summarizes these results. It can be seen that 
in only a few cases could a feature be uniquely determined by this tech- 
nique, and in virtually no case could it be completely delineated. On 
no channel was it possible to unambiguously determine areas of suburban 
development and agriculture. A comparison of results from the four chan- 
nels reveals widely differing assignments of areas to these two catego- 
ries, as well as to the category of "forest.” In several areas, on all 
channels, it was not possible to determine accurately the shoreline of 
the Susquehanna River. Only two orders of streams could be seen: the 

Susquehanna River and major streams entering it. A few lesser streams 
were seen on the original imagery by inspection with the hand lens. 
Subsequent discussion with other researchers indicated that better images 
are obtainable and somewhat greater detail can be mapped on these. 

This study clearly revealed that photointerpretive techniques, 
when applied to ERTS imagery, are unsatisfactory as a single means of 
determining land use categories, for the following reasons: 

1. It is not possible, by the means attempted here, to unam- 
biguously delineate areas of land use categories or water quality. 

2. Establishment of indices for land use categories requires 
planimetry of areas. Where areas cannot be clearly outlined their size 
cannot be accurately determined. 

3. Up to 2 1/2 hours were spent in mapping a small portion 
of the study area in a single channel. Clearly, mapping an entire ERTS 
scene in all four channels would take a large amount of time with very 
limited useful results. 

4. A brief inspection of U2 imagery (flown at 60,000 ft) of 
the same area indicates that some improvement of photointerpretive 



Table 1 : Results of Photointerpretation of ERTS Imagery Using the Saltzman Projector 


Land Use 
Category 

Drainage 


Roads 


Urban 


Suburban 


Forest 


Channel 4 


Incomplete. Is- 
lands obscured. 
Shorelines grade 
into forest. 

Very incomplete. 


Grades into 
suburban. 


Not differen- 
tiable from 
urban. Confused 
with agriculture. 

Not differen- 
tiable from 
drainage and 
often confused 
with agriculture. 


Channel 5 

Channel 6 

Channel 7 

t iciei. i cu 

Channel 

Incomplete. Shore- 
lines grade into 
forest. 

Confused with 
urban. 

Some confusion 
with urban. 

Seven 

Clearly defined 
where white. 
Unrealiable when 
parallel to scan 
lines. Many dark 
lines could be 
roads or drainage. 

Rarely seen and 
poorly defined. 

t 

Rarely seen. 

Five 

Confused with 
probable bare 
fields. Other- 
wise fairly 
distinct. 

Minor confusion 
with suburban 

Confused with 
drainage. 

Five and six 

Not differen- 
tiable from 
agriculture. 

Some confusion 
with drainage. 

Confused with 
agriculture. 

Confused with 
agriculture. 

Fair to poor 
distinction from 
both agriculture 
and urban. 

Confused with 
agriculture. 

All poor, due 
to confusion 
with agri- 
culture. 

Five 


(Continued) 



Table 1 (Continued) 


Land Use 
Category 

Channel 4 

Channel 5 

Channel 6 

Channel 7 

Preferred 

Channel 

Agriculture 

Confused with 
forest and often 
with suburban. 

Not differentiable 
from suburban. 

Confused with 
both forest and 
suburban. 

Confused with 
forest and with 
portions of 
suburban. 

All poor, du< 
to confusion 
with forest 
and suburban 

Construction 

Confused with 
established con- 
crete areas 
(e.g., airport) 
and areas of 
erosion. 

Confused with 
established areas 
of concrete and 
with urban. 

Indistinct. 

1 

Not visible. 

Four. 

Erosion and 
siltation 

Confused with 
construction. 

Not visible. 

Not visible. ' 

Not visible. 

Four. 
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techniques could be realized by using U2 photography to train the photo- 
interpreter to recognize ERTS signatures. This would, however, consid- 
erably increase the time requirements for a given area of investigation. 

5. Better images are now available for the use with the 
Saltzman technique. These images are produced by photographic enhance- 
ment at the expense of radiometric fidelity. However, enhanced images 
are recommended when photointerpretive techniques are required for a 
"first look.” 


Digital Analysis of ERTS-1 Data 

The data used in the digital analysis of the study area was from 
August 1, 1972, scene number 1009—15244. Cloud cover was inconsequential 
over the area of interest. All- four MSS channels were used in data 
processing; however, channel 7 was of poor quality. Using an Ozalid 
print of the channel 7 image as a guide, two subsets of the full scene 
were put on two separate subset tapes, using the SUBSET program 1 . This 
program, in addition to selecting the subsets, reformats the data to be 
compatible with all of the analysis and mapping programs of the ORSER 
data processing system. The first subset was defined as scan lines 937 
through 1150 and elements 2790 through 3010. The second subset consisted 
of lines 1051 through 1200 and elements 3010 through 3228. Both of the 
subsets came from the third tape of the four for the scene. 

The first step in the analysis was the production of a brightness 
map of the area, using the NMAP program, for the purpose of locating 
patterns and targets in the area of interest. This initial output (and 
all subsequent computer map output) was compared with USGS 7 1/2 minute 
quadrangle maps of the area, printed in 1963 and 1969. A uniformity map 
was then produced, and five signature training areas were defined on 
this UMAP output. These areas were: river water, forest, railway yard, 

central urban, and an unknown target which was found to be similar to 
the forest target. The training areas were defined for the STATS pro- 
gram and the mean vectors (spectral signatures) and covariance matrices 
were computed. 


^or complete program descriptions, see ORSER-SSEL Technical 
Report 10-73. 
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Uniform areas could not be found for many targets, tor example, 
clusters of uniform elements were either nonexistent or too small for 
reliable signatures for creek water. The cluster analysis program 
(DCLUS) was modified and used to estimate signatures for creek water, 
roofs (e.g., tops of large building complexes), and highways. The cen- 
tral urban and railroad signatures obtained with training areas were 
verified by cluster analysis. The two methods produced results in per- 
fect agreement. The highway signature obtained by cluster analysis was 
found to be a great deal more general than for highways alone. By using 
the signature for classification and mapping large areas, it was found 
that suburban areas were very well mapped with the highway signature 
and it was therefore renamed. 

After an initial set of signatures was obtained, trial maps of 
blocks of data in the subsets were made. A second stage of target and 
signature determination was begun on the basis of these maps. The areas 
which were unclassified were investigated and, by the use of the methods 
applied before, additional signatures and targets were identified. At 
this stage, training areas were allowed to include a lower level of 
uniformity. The number of observations and the number of subareas with- 
in the training area for each target were, therefore, substantially 
increased to overcome the effect of decreased uniformity. The 7 1/2 
minute quadrangle maps were used in target identification to make sure 
that all subareas included in a training area were of the same target. 

Having obtained these additional signatures, the whole area from 
both subsets was mapped. Only 10 to 15 percent of the area remained 
unclassified. The patterns of unclassified elements appeared to be 
related to non-urban land use, possibly agriculture. One of these areas 
by chance fell within the boundaries of the cluster analysis used for 
the determination of the creek signature. The cluster analysis had 
classified the area homogeneously and the pattern matched the pattern 
of the unclassified area on the large map. The signature for OPEN LAND, 
was taken from that run. Three Mile Island, on the Susquehanna River, 
is mapped on the 1963 Middletown 7 1/2 minute quadrangle sheet as open 
land. However, initial processing indicated that the area is now some- 
thing other than open land, and a check of the underflight photography 
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of the Island revealed the construction of an atomic power plant# A 
cluster analysis was run on the island and surrounding water area, 
resulting in the signature BUILDING. This signature, in addition to 
yielding a classification for Three Mile Island, filled in substantial 
areas in the Harrisburg metropolitan district which had been previously 
unclassified. From the quadrangle maps these areas appeared to consist 
of heavy industry and warehouses. The final map was based on a set of 
thirteen signatures and only three percent of the total area remained 
unclassified. The full set of category information, with signatures, 
is given in Table 2. 

The euclidean distances of separation of categories are given in 
Table 3 . A critical distance of 10.0 was used for every class except 
RIVER WATER, which had a value of 15.0 assigned to it. In the classi- 
fication scheme, an element was assigned to the class for which the 
euclidean distance from it to the class signature was smallest if the 
distance was smaller than the critical distance for the class. If the 
distance was greater than the critical distance, the classification 
would be attempted for the next nearest class, and so on. If the ele- 
ment could not be assigned to any class under these rules, it was 
unclassified. Consider RIVER WATER, for example. The distance of 
separation from each of the other categories is, in every case, greater 
than 15.0. Therefore, there is no chance of confusion between RIVER 
WATER and any other category according to the rules of classification. 
There are a few other categories for which the same is true, based on 
the critical value of 10.0. For most of the classes, however, there 
exist a few distances which indicate potential confusion. Consider 
classes 2 (RAIL) and 5 (URBAN) of Table 3 . The distance of separation 
between these two classes is only 2.3. There is, therefore, a potential 
for confusion between the two classes. 

Three other pairs of classes have small distances of separation 
which should be mentioned. In addition to the aforementioned problem 
with the RAIL signature, this signature also has a relatively small 
distance of separation from the CREEK signature. Whether confusion 
actually exists or not in classifying rail and creek targets can only 
be resolved by ground truth. It is possible that there might be enough 



Table 2: Category Information for the Full Set of Signatures 


Number 


Name 

FOREST 1 

RAIL 

RIVER 

VEGETATION 

URBAN 

GRASS 
FOREST 2 
ROOF 
SUBURB 
PAVEMENT 

CREEK 
OPEN LAND. 
INDUSTRY 

UNCLASSIFIED 


Symbol Limit 

10.0 
10.0 
W 15.0 

10.0 

* 10.0 

10.0 
10.0 
V 10.0 

t 10.0 

@ 10.0 


Count Percent 


29.28 

37.00 

33.18 

31.78 

36.13 

32.83 

28.25 

52.50 

38.74 

40.59 

33.30 

33.40 

42.42 


18.76 

29.45 

22.48 

21.61 

28.25 

22.83 

18.21 

55.00 

31.88 

36.50 

23.52 

22.74 

37.58 


46.68 

29.09 

17.76 

41.06 

29.71 

43.79 

49.54 

56.00 

48.01 
51.95 

31.04 

61.00 

39.20 


27.60 

10.91 

4.78 

22.00 

12.58 

22.50 

29.82 

22.00 

23.88 

25.59 

13.48 

35.23 

15.90 


6844 

5329 

130 

11796 

2738 

1119 

5303 

2241 


vo 


cd cni vo n n 



Table 3: Distances of Separation for Mapping Categories 


Cate- 

gories 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

1 

0.0 

27.6 

37.2 

8.8 

25.5 

8.0 

3.8 

44.4 

16.7 

21.8 

22.0 

17.2 

26.8 

2 

27 . 6 

0.0 

15.1 

18.8 

2.3 

20.3 

31.3 

41.7 

23.1 

28.3 

7.7 

40.8 

14 9 

3 

'37.2 

15 . 1 

0.0 

29.0 

15.7 

31.5 

41.0 

56.5 

37.4 

43.1 

15.9 

52.9 

29.9 

4 

8.8 

18 . 8 

29.0 

0.0 

16.7 

3.2 

12.5 

42.0 

14.3 

20.8 

13.4 

24.0 

20.2 

5 

25.5 

2.3 

15.7 

16.7 

0.0 

18.4 

29.2 

42.0 

'.22.0 

27.4 

5.7 

39,1 

15.1 

6 

8 . 0 

20.3 

31.5 

3.2 

18.4 

0.0 

11.4 

39.6 

11.7 

18.0 

15.6 

21.4 

19 3 

7 

3.8 

31.3 

41.0 

12.5 

29.2 

11.4 

0.0 

45.2 

18.3 

22.6 

25.7 

14.4 

29 6 

8 

44.4 

41.7 

56.5 

42.0 

42.0 

39.6 

45.2 

0.0 

28.1 

22.7 

45.3 

40.1 

26.9 

9 

16.7 

23.1 

37.4 

14.3 

22.0 

11.7 

18.3 

28.1 

0.0 

6.6 

22.3 

20,2 

13 7 

10 

21. 8 

28.3 

43.1 

20.8 

27.4 

18.0 

22.6 

22.7 

6.6 

0.0 

28.4 

20.4 

16 2 

11 

22.0 

7.7 

15.9 

13.4 

5.7 

15.6 

25.7 

45.3 

22.3 

28.4 

0.0 

37.0 

18 8 

12 

17.2 

40.8 

52.9 

24.0 

39.1 

21.4 

14.4 

40.1 

20.2 

20.4 

37.0 

0.0 

33 9 

13 

26.8 

14.9 

29.9 

20.2 

15.1 

19.3 

29.6 

26.9 

13.7 

16.2 

18.8 

33.9 

0.0 
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sediment, low vegetation, and water in the yards to give a true response 
for the creek classification. The two other pairs of categories with 
small separation distances are CREEK with URBAN, and HIGHWAY with SUBURB. 
The reason for the similarity of CREEK and URBAN signatures is not known 
at this time. The similarity of the HIGHWAY and SUBURB signatures was 
not unexpected, because the initial highway signature was renamed SUBURB 
when it gave very good mapping results for suburban areas. The new 
highway signature was obtained later on and may indeed have also been 
based on targets very similar to those of the suburban signature. It 
seems, however, that the new highway signature is more related to park- 
ing lots and similar paved and unpaved areas than it is to the suburban 
signature. Actual highways are mapped by both symbols. 

Some serious problems exist in naming the categories. Some of them 
are easily named correctly, such as RIVER WATER and FOREST. Little 
emphasis was put on the names of other categories because they were 
named only inf erentially , with no direct means of being sure of the 
targets. It is not at all an easy matter to pick out vegetation signa- 
tures in ERTS data simply by looking at the signatures. It is even more 
difficult to identify other signatures. The quadrangle maps are of 
limited utility since they do not generally give the kind of information 
needed to identify a category except on an inferential basis. Ground 

or aerial photographs, such as used in identifying the construction 
on Three Mile Island, would have been very helpful in specifically iden- 
tifying and naming the targets. 

A small part of the DCLASS map is shown in Figure 1 . The river 
islands in the river are readily apparent. The central metropolitan 
area of Harrisburg, mapped with * T s, can be seen in the upper right 
portion of the figure. Heavy industrial and warehouse areas, mapped 
with + f s, can be seen adjoining the downtown area of Harrisburg. Across 
the river, the Camp Hill suburban area can be seen mapped with //'s. The 
® ® Camp Hill possibly Indicate parking lots or bare ground, and the 
- r s indicate parks, cemeteries, and similar green areas. A summary of 
the mapping results is included in Table 2 . The percentage in each 
category is the relative acreage in the category. The count for each 
category is the actual number of elements classified in the category. 
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The conversion factor to acreage is approximately 1.12 acres/element, 
based on the distance of separation between elements in a line and 
between lines. 

The results of this study amply demonstrate that ERTS-1 data can 
be translated to maps using only USGS 7 1/2 minute quadrangle maps for 
reference. The computer generated maps agree quite well with the quad- 
rangle maps except that the fine detail of the latter maps cannot be 
achieved with ERTS-1 data. The ERTS data based maps indicate that more 
significant land use categories can be mapped than has been done on the 
quadrangle maps. In addition, the obvious and serious deficiency of 
the quadrangle maps is very strongly demonstrated: they are obsolete 

in many areas, even over the short period of time since the 1969 publi- 
cation dates. The maps made in 1963 are of exceedingly limited utility 
in areas where rapid transitions in land use are in evidence. Because 
of their obsolescence and the absence of sufficient land use classifi- 
cation categories, the use of USGS quadrangle maps alone to support 
ERTS data based mapping is definitely inadvisable. ' Underflight photog- 
raphy or imagery, and photointerpretation of these, are, without ques- 
tion, a needed basis of support for digital mapping of ERTS data. Some 
timely ground truth is also necessary to resolve anomalies. It was 
found that very little else could be done without such additional sup- 
port. Signatures could be refined and the number of signatures could 
be increased, but there was no justification for such additional work 
since the end result would be the same, in that the interpretation would 
still lack sufficient support. 


Combined Techniques 

After separate analysis of ERTS-1 data by photointerpretation alone 
and by computer processing of MSS digital data without the assistance of 
photointerpretation, it became apparent that each method had shortcomings 
which might be overcome if the methods were combined. Computer differ- 
entiation of areas from scanner data is far superior to that done by the 
human eye. Computation of areas from the digital data makes delineation 
of these areas unnecessary and is far more accurate than planemetric 
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methods at the scale of ERXS MSS imagery. Since the end result of 
processing ERTS— 1 data is a map, the automated processes of thematic 
mapping by computer is the efficient way to go. However, "ground truth" 
is the key to correct signatures for this mapping. Underflight data and 
photointerpretation of underflight photography, as well as of ERTS 
imagery, are vital links leading to valid signatures for the thematic 
map. A marriage of these two disciplines, photointerpretation and com- 
puter processing, is essential for maximum utilization of ERTS-1 data. 
The two analysis teams, therefore, combined forces and evolved a method 
of ERTS MSS data analysis referred to as the "hybrid approach" and shown 
Figure 2 . This method involves intimate interaction of the computer 
analyst and the photointerpreter, using high-altitude photography (U2) 
for comparison with the computer output. The hybrid approach was first 
successfully applied in a land use study of the Harrisburg area origi— 
nal ly examined by the two separate methods. This study is discussed in 
ORSER-SSEL Technical Report 14-73. 


Conclusions 

ORSER feels strongly that a hybrid approach is essential to ERTS 
data analysis. Computer differentiation of areas from scanner data is 
far superior to that done by the human eyej but the photointerpreter, 
working with underflight photography, is an essential and integral part 
of data processing for providing identification of features exhibited on 
computer output. 

The results of this investigation has shown that ERTS digital data 
can be translated into map form using only USGS maps for reference. 
However, such maps are frequently insufficient to provide enough infor- 
mation for classification, particularly in areas where rapid transitions 
in land use are evident and maps become obsolete very quickly. It has 
also been shown that photointerpretation techniques can be applied to 
ERTS imagery. However, in only a few cases could a feature be uniquely 
determined by this method alone. The use of U2 and C130 imagery has 
been found to improve these interpretation results, but photointerpretive 
techniques have not been completely satisfactory as a single means of 
analysis. 



Explanation of the steps shown in Figure 2. 


PRELIMINARY PROCEDURES 

A* 1 Determine scan line and element limits. 

B. This becomes the working tape. 

C. Identify clouds. 

D. Review scene for definable boundaries. 

FIRST LEVEL MAPPING 

A. Collaboration of the photo interpreter and 
computer mapper. Select easiest targets 
first. Choose spectrally homogeneous items 
with positive geographic locations. Select 
replications in widely separated areas. 

B. Identify some targets (training areas) on 
NMAP or UMAP. 

C. Check for uniformity on UMAP. Attempt to 
find a large number of like elements. Loop 
A, B, and C until a sufficient number of 
training areas are identified. 

D. Review statistical characteristics of 
defined targets. 

E. Make first run on classification map. 

F. This is a verification step. Project U2 
image onto computer map. Identify satis- 
factory classifications. If some areas 
lack definition, redefine training areas. 


SECOND LEVEL MAPPING 

A. Attempt to identify items outside training 
areas . 

B. Define items not subject to definition by 
training areas. These might be linear 
features or stream channels. Add these 
to the list of signatures and continue. 

C. This a recycle, with smaller training 
areas and more weight placed on cluster 
analysis . 

THIRD LEVEL MAPPING 

A. Review the classification categories origi- 
nally defined as desirable. If present map 
output is unnecessarily refined, combine 
some groups . 

B. Some categories will require broadened spec- 
tral parameters. A series of successive 
approximations will be required to define 
these units. The resulting training areas 
will be less spectrally homogeneous. 

C. Requires collaboration of the photointer- 
preter and the computer mapper. 

D. Establish limiting goal. 


^he letters correspond directly to those shown in the diagram. 
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Figure 2 : Flow diagram, for the hybrid approach to ERTS data processing. 
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